Action Planning Device

ABSTRACT

Vehicle systems that implement action planning for a vehicle have had difficulty in maintaining safety in the event of a failure in a recognition device since these systems generate trajectories and control the vehicle on the basis of outside-world information recognized by recognition devices. The present invention was conceived in light of the aforementioned situation and addresses the problem of maintaining safety even in the event of a failure in a recognition device of a vehicle system. This problem can be solved by an action planning device having a failure detection unit for detecting a failure in an outside-world recognition unit and a trajectory generation/determination unit for executing an action on the occurrence of a failure on the basis of outside-world information outputted by the outside-world recognition unit, wherein the trajectory generation/determination unit takes an action for coping with the failure on the basis of failure information from the failure detection unit.

TECHNICAL FIELD

The present invention relates to an action planning device and an actionplanning system.

BACKGROUND ART

As a background art of the present technical field, there is JP2010-287093 A (PTL 1). In this publication, an object is to provide “acourse evaluation device and a course evaluation method that can improverunning efficiency, can avoid interference with other object, and canevaluate a course of a moving object while realizing both the runningefficiency and the interference avoidance” and “a course generation unit11 generates a plurality of prediction courses of an own vehicle on thebasis of running information. A negative course generation unit 12generates negative courses for the plurality of prediction courses. Apedestrian approaching action calculation unit 13 acquires a pedestrianapproaching course on the basis of running information transmitted froma running information acquisition unit 2. A negative course evaluationunit 14 compares the plurality of negative courses and the pedestrianapproaching course and generates a negative evaluation value for eachnegative course. The course evaluation unit 15 determines the predictioncourse having a largest negative evaluation value as a course of the ownvehicle” is described as a resolving means.

As another background art, there is JP 2010-211301 A (PTL 2). In thispublication, an object is to provide “an accidentprediction/notification device, an accident prediction/notificationsystem, and an on-vehicle device that cope with a collision accident ofpedestrians when turning left at an intersection and predict an areawhere an accident may occur in consideration of past traffic pathhistories of pedestrians and vehicles” and “when a vehicle entering theintersection arrives at a designated point A1, an accident occurrenceprediction area is predicted on the basis of pedestrian information nearthe intersection when the entering vehicle arrives at the designatedpoint A1, signal light information of a traffic light 5, and a learningarea map stored in a storage unit 37 and the accident occurrenceprediction area is transmitted as an accident occurrence prediction areamap to the on-vehicle device 61. The on-vehicle device 61 displays thereceived accident occurrence prediction area map on a liquid crystaldisplay panel of the on-vehicle device 61 and performs notification to avehicle 6 having the on-vehicle device 61 mounted thereon.” is describedas a resolving means.

CITATION LIST Patent Literature

PTL 1: JP 2010-287093 A

PTL 2: JP 2010-211301 A

SUMMARY OF INVENTION Technical Problem

In the course generation of PTL 1, for a generation method of atrajectory securing safety, a method of securing the safety by negativeevaluation in particular is described. However, an operation when afailure occurs in a recognition device is not described.

In addition, in PTL 2, a method of determining a risk position whencommunication is interrupted and generating a warning is described.However, a generation method of a trajectory securing safety when afailure occurs in a recognition device mounted on a vehicle is notdescribed.

The present invention has been made in view of the above circumstancesand provides a method of enabling a vehicle system to take a safeaction, even when a failure occurs in a recognition device andoutside-world information cannot be acquired.

Solution to Problem

To solve the above problem, an embodiment of the present invention mayuse a technical spirit described in claims, for example.

Advantageous Effects of Invention

According to the present invention, even when a failure occurs in arecognition function of a vehicle system, the vehicle system can take asafe action. Particularly, even when a dynamic object exists until apoint of time of occurrence of the failure in a failure occurrence rangeof a recognition device, an action such as predicting an action,maintaining a safe state, and delivering an operation to a user isenabled.

In addition, according to a different embodiment, action prediction ofsurrounding dynamic objects (a vehicle, a two-wheeled vehicle, and apedestrian) is performed safely, so that a safe action can be taken whenthe failure occurs in the recognition function.

In addition, according to a different embodiment, action planning andcontrol are performed such that the dynamic object is hard to enter anarea where the failure occurs. As a result, the risk of the dynamicobject entering the failure occurrence range of the vehicle can beavoided.

In addition, according to a different embodiment, even when the dynamicobject enters from a dead angle range of the recognition device as aresult of the occurrence of the failure, a safe action can be taken.

In addition, according to a different embodiment, when there is arecognition device recognizing the same range as the failure occurrencerange, a result of prediction based on past information of therecognition device in which the failure occurs and a recognition resultof a normal recognition device are superimposed. As a result, theexistence probability of an obstacle can be acquired with highprecision.

In addition, according to a different embodiment, in the case where thefailure occurs in the recognition device when a traffic lane changeoperation is executed, an action according to the failure occurrencerange of the recognition device and a current control state is taken. Asa result, a safe action can be taken.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a flowchart illustrating trajectory generation processing inan action planning system.

FIG. 2 illustrates an example of a system.

FIG. 3 illustrates an example of an internal configuration of a vehiclecontrol system.

FIG. 4 illustrates a configuration example of a controller.

FIG. 5 illustrates an example of a software module configuration of acontroller.

FIG. 6 illustrates a configuration example of an action planning system.

FIG. 7 illustrates an arrangement example of an action planning systemin a vehicle control system.

FIG. 8 illustrates an example of outside-world recognition.

FIGS. 9(a) and 9(b) illustrate an example of an outside-worldrecognition map.

FIG. 10 illustrates an example of a list type of an outside-worldrecognition map.

FIG. 11 illustrates an example of trajectory generation based onoutside-world recognition map information.

FIGS. 12(a) to 12(d) illustrate an example of an outside-worldrecognition map and trajectory generation at the time of failureoccurrence.

FIG. 13 illustrates an output example by an output device.

FIGS. 14(a) and 14(b) illustrate an example of an outside-worldrecognition map and trajectory generation at the time of failureoccurrence in accordance with a second embodiment of the presentinvention.

FIGS. 15(a) and 15(b) illustrate an example of an outside-worldrecognition map and trajectory generation at the time of failureoccurrence in accordance with a third embodiment of the presentinvention.

FIGS. 16(a) to 16(d) illustrate an example of an outside-worldrecognition map and trajectory generation at the time of failureoccurrence in accordance with a fourth embodiment of the presentinvention.

FIGS. 17(a) to 17(c) illustrate an example of an outside-worldrecognition map and trajectory generation at the time of failureoccurrence in accordance with a fifth embodiment of the presentinvention.

FIGS. 18(a) to 18(d) illustrate an example of an outside-worldrecognition map and trajectory generation at the time of failureoccurrence in accordance with a sixth embodiment of the presentinvention.

DESCRIPTION OF EMBODIMENTS

Hereinafter, preferred embodiments of the present invention will bedescribed. This embodiment mainly describes an action planning device ofa vehicle control system in a vehicle system and is suitable forexecution in the vehicle system. However, this embodiment does notdisturb an application to any system other than the vehicle system.

First Embodiment <Configuration of Vehicle Control System>

FIG. 2 illustrates an outline of a system having an action planningdevice according to this embodiment. 1 shows a vehicle system having avehicle control system in a vehicle. 2 shows a vehicle control systemconfigured using an on-vehicle network (CAN: Controller Area Network,CANFD: CAN with Flexible Data-rate, and Ethernet (registered trademark))and a controller (ECU: Electronic Control Unit), for example. 3 shows acommunication device that performs wireless communication (for example,communication of a mobile phone and communication using protocols of awireless LAN and a WAN) with the exterior of the vehicle system 1,performs wireless communication to acquire or transmit informationregarding an outside world (infrastructures and other vehicles) orinformation regarding an own vehicle, performs wired connection with adiagnostic terminal (OBD), an Ethernet terminal, and an externalrecording medium (for example, a USB memory and an SD card) terminal,and performs wired communication with the vehicle control system 2. 4shows a vehicle control system configured using a network using aprotocol different from or equal to a protocol of the vehicle controlsystem 2. 5 shows a drive device such as an actuator that drivesmechanical and electrical devices (for example, an engine, atransmission, a wheel, a brake, and a steering device) to control avehicle motion, according to control of the vehicle control system 2. 6shows a recognition device that is configured using outside-worldsensors such as a camera, a radar, LIDAR, and an ultrasonic sensor toacquire information input from the outside world and generateoutside-world recognition information to be described below and dynamicsystem sensors (acceleration, wheel speed, and GPS: Global PositioningSystem) to recognize a state (a motion state and a position state) ofthe vehicle system 1. 7 shows an output device such as a liquid crystaldisplay, a warning light, and a speaker that is connected to a networksystem by wire or wireless, receives data transmitted from the networksystem, and displays or outputs necessary information such as messageinformation (for example, video and audio). 8 shows an input device suchas a steering, a pedal, a button, a lever, and a touch panel to generatean input signal to allow a user to input an intention or an instructionof an operation to the vehicle control system 2. 9 shows a notificationdevice such as a lamp, an LED, and a speaker to allow the vehicle system1 to notify the outside world of a state of the vehicle.

The vehicle control system 2 is connected to the vehicle control system4, the communication device 3, the drive device 5, the recognitiondevice 6, the output device 7, the input device 8, and the notificationdevice 9 and performs transmission and reception of individualinformation.

FIG. 3 illustrates a hardware (H/W) configuration example of the vehiclecontrol system 2. 301 shows a network link that connects network deviceson an on-vehicle network. As an example of the network link 301, thereis a network link such as a CAN bus. 302 shows an ECU that is connectedto the network link 301, the drive device 5, the recognition device 6,and a network link (including a dedicated line) other than 301 andcontrols the drive device 5 or the recognition device 6, acquiresinformation, and performs transmission and reception of data with thenetwork. 303 shows a gateway (hereinafter, referred to as the GW) thatconnects a plurality of network links 301 and performs transmission andreception of data with the individual network links 301.

In this embodiment, a bus-type network topology in which a plurality ofECUs 302 are connected to two buses is described. However, a star-typenetwork topology in which the plurality of ECUs 302 are connected inseries to the GW 303, a link-type network topology in which the ECUs 302are connected to a series of links in a ring shape, or a mix-typenetwork topology in which individual types are mixed and which isconfigured using a plurality of networks can be adopted. For the GW 303and the ECU 302, an ECU having a GW function and a GW having a functionof the ECU exist.

The ECU 302 executes control processing such as an output of a controlsignal to the drive device 5, acquisition of information from therecognition device 6, an output of a control signal and information tothe network, and a change of an internal state, on the basis of datareceived from the network.

FIG. 4 illustrates an example of an internal configuration of the ECU302 or the GW 303 to be a network device according to the presentinvention. 401 shows a processor such as a CPU that has a storageelement such as a cache and a register and executes control. 402 showsan input/output (I/O) that performs transmission and reception of datato the network link 301 or the drive device 5 or/and the recognitiondevice 6 connected by a network or a dedicated line. 403 shows a timerthat performs management of time using a clock not illustrated in thedrawings. 404 shows a read only memory (ROM) that stores a program andnon-volatile data. 405 show a random access memory (RAM) that storesvolatile data. 406 shows an internal bus that is used for communicationin the ECU.

Next, a configuration of a software module operated by the processor 401is illustrated in FIG. 5. 502 shows a communication management unit thatmanages an operation and a state of the I/O 402 and sends an instructionto the I/O 402 via the internal bus 406. 503 shows a time managementunit that manages the timer 403 and performs information acquisition orcontrol for the time. 501 shows a control unit that performs analysis ofdata acquired from the I/O 402 or whole control of the software module.504 shows a data table that holds information such as an outside-worldrecognition map to be described below. 505 shows a buffer thattemporarily secures data.

For the configuration of FIG. 5, an operation concept on the processor401 is illustrated. Information necessary for the operation isappropriately acquired from the ROM 404 and the RAM 405 or isappropriately written to the ROM 404 and the RAM 405 and the operationis executed.

Each function of the vehicle control system to be described below isexecuted by the control unit 501.

<Functional Configuration Example of Vehicle Control System>

A functional configuration example of the vehicle control system isillustrated in FIG. 6.

601 shows an entire portion of an action planning system according tothe present invention. 602 shows an integration recognition unit thatintegrates outside-world recognition information output from theplurality of recognition devices 6, generates an outside-worldrecognition map to be described below, has a failure detection unit 608to be described below, and generates the outside-world recognition mapat the time of occurrence of a failure to be described below. 603 showsa trajectory generation/determination unit that performs determinationfrom the outside-world recognition map generated by the integrationrecognition unit 602 and an input of a user input from a user input unit605, generates a trajectory, sends a motion control instruction to amotion control unit 604, sends an output instruction to an outputmanagement unit 606, and sends a notification instruction to anotification management unit 607. 604 shows a motion control unit thatcontrols the plurality of drive devices 5, according to the motioncontrol instruction from the trajectory generation/determination unit603. 605 shows a user input unit that generates instruction informationof the user, according to an input from the input device 8. 606 shows anoutput management unit that sends an output instruction to the outputdevice 7, according to an output of the trajectorygeneration/determination unit 603. 607 shows a notification managementunit that sends a notification instruction to the notification device 9,according to the output of the trajectory generation/determination unit603. 608 shows a failure detection unit that detects a failure occurringin the recognition device 6 or a failure occurring in a communicationpath between the recognition device 6 and the integration recognitionunit 602.

All or any combination of the integration recognition unit 602, thetrajectory generation/determination unit 603, the motion control unit604, the user input unit 605, the output management unit 606, and thenotification management unit 607 is called an action planning system anda part or all of the drive device 5, the recognition device 6, theoutput device 7, the input device 8, and the notification device 9 maybe included in the action planning system.

The action planning system 601 is configured using a plurality offunctions and a plurality of patterns exist in an arrangement offunctions in the H/W illustrated in FIG. 3. An example of thearrangement is illustrated in FIG. 7. The arrangement of the functionsis not limited thereto and the individual functions may be disposed inECUs different from the ECUs described. For example, the integrationrecognition unit 602 and the trajectory generation/determination unit603 may be disposed in the same ECU. The functions are disposed in thesame ECU, so that communication between the functions is facilitated,and high-speed processing can be realized. In addition, the functionsare disposed in the different ECUs, so that the individual functions areprotected from the risk of a common cause failure by an H/W failure, andhigh reliability can be realized. The individual functions will bedescribed below.

<Outside-World Recognition Method>

The kinds of the recognition devices 6 are as described in theconfiguration of the vehicle control system and outside-worldrecognition information to be described below is acquired by theoperation principle according to the kind of each recognition device.Mainly, sensors of the recognition device 6 acquire physical measurementvalues of the outside world, apply a specific algorithm (for example, animage recognition algorithm for an acquired image) to the measurementvalues, and acquire outside-world information.

For each recognition device 6, a measurable range is previouslydetermined. For example, in the case of a camera, a recognition limit ofa distant distance is determined by a photographing direction,vertical/horizontal angles, and the number of pixels. Meanwhile, in thecase of radar, a radiation angle and a reception angle of radio wavesand a distance are determined. In addition, the measurable range ismeasured by adjusting (calibrating) a change according to an environmentand the measurable range of the recognition device 6 is determined. Anoutside-world situation of the vehicle system 2 can be confirmed bycombining the outside-world information acquired by the individualrecognition devices 6.

An example of the outside-world recognition is illustrated in FIG. 8.Here, an example of the case where the recognition devices 6 disposed infour directions of the vehicle system 1 acquire the outside-worldinformation is illustrated. From the outside-world recognitioninformation output from the recognition devices 6, the integrationrecognition unit 602 can confirm an object existing around the vehicle.

The outside-world recognition information can be acquired equally fromthe communication device 3. The information acquired from thecommunication device 3 is mainly information that cannot be observed bythe recognition device 6, for example, outside-world recognitioninformation and position information of an object existing at anopposite side of a shield material such as a shade. The vehicle system 1can confirm a position where the object exists, using the informationacquired from the communication information 3.

In the outside-world recognition, the recognition devices 6 cannotsurely recognize all objects and there is an area (undecided area) wherean existing object is undecided. In this case, uncertainty andreliability of existence of an object are expressed by the existenceprobability to be described below.

<Outside-World Recognition Information>

The outside-world recognition information becomes information thatexpresses objects existing at the exterior of the vehicle system andobserved by the recognition device. Examples of the outside-worldrecognition information include types of objects (stationary objects(walls, dividing lines, signals, separation zones, tree, or the like),dynamic objects (pedestrians, vehicles, two-wheeled vehicles, bicycles,and the like), whether running (area entry) is enabled, and otherattribution information), relative position information(directions/distances) of objects, absolute position information(coordinates and the like) of the objects, speeds, directions (movementdirections and face directions), accelerations, and existenceprobabilities (likelihoods) of the objects, measurement time of theoutside-world recognition information, IDs of recognition devicesexecuting measurements, and the like.

As a calculation method of the existence probability, there is a methodof increasing the probability at the time of determination aslikelihood, including a time-series observation result (the same type ofobject exists at the same position in short time) in addition to anoperation result of the probability by an object determination algorithmin the recognition device 6. In this way, the possibility that theobject can be correctly determined by the observed outside-worldrecognition information can be increased.

In addition, measurement time of the outside-world recognitioninformation is held, so that an outside-world recognition map where aplurality of outside-world recognition information is synchronizedtemporally can be generated. Even in the case where a failure hasoccurred in the recognition device 6, when the outside-world recognitioninformation is not updated thereafter, a final state (a final positionand a final observation time) before the failure occurs is grasped andthe following action can be predicted. When the failure occurrence timeof the recognition device 6 is known, only the outside-world recognitioninformation before the failure occurs in the outside-world recognitioninformation generated by the recognition device 6 in which the failurehas occurred can be trusted.

In addition, a recognition device ID showing the recognition device 6having generated respective outside-world recognition information isincluded in respective outside-world recognition information. As aresult, when the failure has occurred in any recognition device, it canbe confirmed which outside-world recognition information is output fromthe recognition device in which the failure has occurred.

<Outside-World Recognition Map>

The integration recognition unit 602 generates integration recognitioninformation (example: outside-world recognition map) obtained byintegrating the outside-world recognition information received from theplurality of recognition devices 6. An example of the outside-worldrecognition map is illustrated in FIGS. 9(a) and 9(b). Here, an exampleof the case where object information is disposed for each area withrespect to an orthogonal coordinate system (grid) (FIG. 9(a)) isillustrated in FIG. 9(b). The object information is content in the caseof removing position information from the example of the outside-worldrecognition information and is disposed in each grid.

In the case where a plurality of object information exist in the samegrid (for example, in the case where a plurality of recognition devicesobserve a position of the same grid), for example, when recognition isenabled from the plurality of recognition devices, the existenceprobability is increased and when recognition is disabled from theplurality of recognition devices observing the same grid, the existenceprobability is decreased. As a result, precision of the recognition canbe improved. When information is mismatched, the outside-worldrecognition information having the high probability is preferentiallyused and the existence probability of the object information in the gridis decreased. Thereby, when different information is recognized by theplurality of recognition devices 6, the existence probability isdecreased and reliability of the object information can be decreased.

As a different expression of the outside-world recognition map, there isa list-type method for performing listing for each recognized object, inaddition to a notation using the grid. An example of a list-typenotation is illustrated in FIG. 10. 1001 shows the entire outside-worldrecognition map by list display. As such, the outside-world recognitionmap is held in the list type, so that a data amount can be reduced ascompared with the grid type.

<Action Prediction>

The outside-world recognition map can be generated by performingprediction (action prediction) from past outside-world recognitioninformation as well as the entire outside-world recognition informationobserved at the present time. For example, in the case of a stationaryobject, the stationary object is likely to exist at the same position(the same position on a road surface, not a relative position with avehicle) after constant time passes and in the case of a dynamic object,a position after the constant time can be predicted from immediatelyprevious position, speed, and acceleration. As such, the predictedoutside-world recognition information is used, so that information of aposition where an observation is disabled at the present time can bepredicted.

The integration recognition unit 602 can perform the action predictionon the basis of the outside-world recognition map. However, therecognition device 6 may add future prediction information to theoutside-world recognition information, may transmit the outside-worldrecognition information, and may notify the integration recognition unit602 of the outside-world recognition information. In this case, eachrecognition device 6 performs prediction and an operation amountassociated with the action prediction of the integration recognitionunit 602 can be reduced. In a different method, the trajectorygeneration/determination unit 603 may perform action prediction of anecessary object from a current outside-world recognition map. In thisway, a communication load from the integration recognition unit 602 tothe trajectory generation/determination unit 603 can be reduced andaction prediction of only an object necessary for generating anddetermining the trajectory can be performed.

<Trajectory Generation>

A trajectory generation method based on the outside-world recognitionmap will be described. The trajectory is generated to satisfy safetyrestrictions in which the vehicle system 1 can run safely (example: thepossibility of colliding with other obstacle is low) and motionrestrictions to be acceleration, deceleration, and a yaw rate which thevehicle system 1 can realize.

In the outside-world recognition map of the example of FIG. 9(b), ageneration example of a trajectory where an own vehicle moves to a righttraffic lane will be described using FIG. 11. Here, an example of thecase where a running vehicle exists on the right traffic lane, but aspeed of the own vehicle is higher than a speed of the running vehicleand a traffic lane can be changed is illustrated. First, the own vehiclegenerates a trajectory that satisfies the motion restrictions and movesto the right traffic lane. Then, occurrence or non-occurrence ofcollision by a prediction trajectory (for example, a position afterconstant time at a current speed and assumed acceleration) of otherdynamic object and the trajectory of the own vehicle is calculated forthe generated trajectory. When non-occurrence of the collision iscalculated, the own vehicle is controlled on the basis of the trajectoryof the own vehicle. When occurrence of the collision is calculated,recalculation is performed after constant standby time or a differenttrajectory satisfying the motion restrictions is generated and thesafety restrictions are calculated in the same way.

Calculation methods of the safety restrictions include a potential mapmethod of calculating a risk of each area from a type, a speed, and amovement direction of each object and calculating risk potential, inaddition to a method (entry prohibition area method) of setting an areaassumed from the current speed and the assumed acceleration/decelerationof the dynamic object as an entry prohibition area, as described above.When the potential map method is used, trajectories having lowestpotential and not entering a potential area of a constant value or morein a generated potential map are generated and a trajectory satisfyingthe motion restrictions of the own vehicle is set as a generatedtrajectory.

For the entry prohibition area, the action prediction of the dynamicobject is necessary. For the action prediction, there is a method ofsetting a constant area based on a movement point of the dynamic objectat the current speed/acceleration and in the current direction as theentry prohibition area. As such, the constant area is set as the entryprohibition area, so that an operation by complex prediction becomesunnecessary.

As such, the trajectory is generated on the basis of the movementdirection of the vehicle, the motion restrictions, and the safetyrestrictions, the trajectory generation/determination unit 603 transmitsthe trajectory information to the motion control unit 604 on the basisof the generated trajectory, and the motion control unit 604 controlsthe drive device 5 on the basis of the trajectory information andcontrols the motion of the vehicle system 1.

<Failure Determination>

A determination method of the failure occurrence will be described. Theintegration recognition unit 602 performs communication with therecognition devices 6 via a network or a communication path such as adedicated line and determines presence/absence of the failure in thecommunication. For a failure of the communication path, the failure ofthe communication path can be determined by non-performance ofcommunication (error response of communication processing andabnormality of a potential of a signal line) and abnormality of a signalvalue of the communication (example: mismatching of CRC and mismatchingof a fixed data pattern). For the failure of the communication path, acommunication path other than the communication path to transmit theoutside-world recognition information is further provided and occurrenceof the failure of the communication path to transmit the outside-worldrecognition information can be notified by the path.

In addition, examples of the failure of the recognition device 6 includenon-arrival of data transmitted at a constant cycle, late arrival of thedata, non-reception of a fixed data pattern (a head bit is always 1 andCRC is mismatched), and non-response for a request for datatransmission.

In addition to the determination method, the failure can be determinedon the basis of the behavior of transmitted data. For example, for theoutside-world recognition information, when the object executes anon-assumed operation (a result not assumed as an output of therecognition device 6, for example, acquisition of information moving ata movement speed beyond a physical limit of an object type), when anobject beyond a probability range appears or disappears, or when anon-regulated parameter is acquired, that is, when abnormality occurs inthe behavior, the recognition device 6 having outputted theoutside-world recognition information of the object having the abnormalbehavior is handled as a recognition device in which the failure hasoccurred and determination of a failure range to be described below isperformed.

In addition, the recognition device 6 may notify the integrationrecognition unit 602 that the recognition device 6 has failed.Particularly, when a failure of a recognition function of a specificarea (abnormality of a sensor device) has occurred and when therecognition device 6 can determine the failure, the recognition device 6can transmit that the specific area has failed. In this case, thespecific area can be used for the determination of the failure range tobe described below.

By a failure determination result, the recognition device 6 in which thefailure occurs can be determined and a failure occurrence range in whichrecognition is disabled can be specified.

Particularly, when the occurrence of the failure is detected by thebehavior, as failure occurrence time, it is determined that the failureoccurs before receiving the outside-world recognition information inwhich the behavior is abnormal and the following outside-worldrecognition information is discarded not to be used for control of thevehicle system 1. As a result, incorrect trajectory generation byincorrect outside-world recognition information after the occurrence ofthe failure can be prevented.

For the failure occurrence time of the recognition device 6, it may beassumed that the failure occurs before constant time from the failuredetection and processing may be executed, at the time of a design. Forexample, detection time of the failure by the failure determinationmethod (for example, interruption of communication) is set as T, T−a (ais a design value: for example, assumption time until the communicationis interrupted after the failure occurs) is set as the failureoccurrence time, and outside-world recognition information output fromthe recognition device 6 in which it is recognized that the failure hasoccurred after the failure occurrence time is completely discarded. As aresult, time until the recognition device 6 detects the occurrence ofthe failure and performs the notification can be increased (a failuredetection processing load decreases) and the integration recognitionunit 602 can be prevented from executing processing on the basis oferroneous outside-world recognition information.

<Trajectory Generation Flow Based on Failure Occurrence State>

A trajectory generation method of the action planning system 601according to the present invention based on the failure occurrence statewill be described using FIG. 1.

The integration recognition unit 602 acquires the outside-worldrecognition information from the recognition device 6 (S101). After S101or at the same time, failure determination is performed and it isdetermined whether a failure occurs in the recognition device 6 or acommunication path with the recognition device 6 (there is a failure ina recognition function) (S102). When it is determined that there is nofailure in the recognition function, on the basis of S102 (no of S103),as usual, trajectory generation processing is executed. For this reason,the integration recognition unit 602 generates an outside-worldrecognition map from the outside-world recognition information andtransmits the outside-world recognition map to the trajectorygeneration/determination unit 603. The trajectorygeneration/determination unit performs trajectory generation by thetrajectory generation method, on the basis of the outside-worldrecognition map (S106). Meanwhile, when it is determined that thefailure occurs in the recognition function, on the basis of S102 (yes ofS103), the integration recognition unit 602 determines a failureoccurrence range by a mechanism described in the failure determinationmethod and generates an outside-world recognition map including thefailure range (S104). After S104, the integration recognition unit 602transmits the generated outside-world recognition map to the trajectorygeneration/determination unit 603. The trajectorygeneration/determination unit 603 generates a trajectory to cope withthe failure of the recognition function by a method of generating atrajectory to cope with the failure to be described below, on the basisof the outside-world recognition map received from the integrationrecognition unit 602 (S105). In this way, when the failure occurs in therecognition function, a trajectory based on the failure range of therecognition function can be generated.

Here, for the failure occurrence range, outside-world recognitioninformation after failure occurrence time is discarded and theoutside-world recognition information is not used in trajectorygeneration based on a failure generation state, so that the trajectorycan be avoided from being generated erroneously on the basis of theoutside-world recognition information after the failure occurrence.

<Generation of Trajectory to Cope with Failure>

A specific example of an outside-world recognition map at the time ofoccurrence of a failure is illustrated in FIGS. 12(a) to 12(d). Here, anexample of the case where a failure occurs in the recognition device 6monitoring a right side of the vehicle system is illustrated (a shadedportion of FIG. 12(a)). In this case, because there is a dynamic objectobserved before failure occurrence in a failure occurrence area, for anaction thereof, action prediction from a point of time of finalobservation before the failure occurrence is performed and a trajectoryto enter a safe state when the failure occurs to be described below isgenerated on the basis of an outside-world recognition map including aresult of the action prediction of the failure occurrence range.

As an example of a trajectory generated by the trajectorygeneration/determination unit, because a failure does not occur at afront side, a trajectory to move to the front side is generated orbecause a failure does not occur in the recognition device of the leftside, a trajectory to move to the left side, confirm a safe area wherethere is no obstacle, and stop at the safe area is generated.

Likewise, an operation of the case where a failure occurs in therecognition device of the left side of the vehicle is illustrated inFIG. 12(c). Here, the case where the left side of the vehicle cannot berecognized is illustrated. Even in this case, a trajectory to enter asafe state when a failure occurs to be described below is generated inthe same way. For example, because a failure does not occur at a frontside, a trajectory to move to the front side is generated or becausethere is no dynamic object before the failure occurrence in a failureoccurrence range of the recognition device of a left side, the left sideis determined as safe and a trajectory to move to the left side, confirma safe area where there is no obstacle, and stop at the safe area isgenerated. In this way, generation of a trajectory to cope with thefailure occurrence range is performed, the risk existing in the failuregeneration range is avoided, and a safe operation can be executed in aconfirmable range.

Here, an example of a grid type is illustrated as an expression methodof the failure occurrence range. However, even in a list type, a type ofan object is set as a failure range and a range is set as a range wherea failure occurs, so that the failure occurrence range can be expressedin the same way.

For the outside-world recognition map after the failure occurrence,information is updated whenever constant time passes, the actionprediction of the dynamic object existing in the failure occurrencerange is performed, the information is combined with the outside-worldrecognition information output from the normal recognition device, andan outside-world recognition map is generated again on the basis of anaction prediction result. In this way, the action of the failureoccurrence range after the constant time passes can be predicted again.

<Safe State when Failure Occurs>

An example of a safe state (safe state when a failure occurs) when thefailure occurs is illustrated below. Examples of the safe state includea state in which there is not the possibility that collision occurs in aprediction range or control is delivered to a user in a safe state.

As one example, the own vehicle stands still in an area (for example, aroad shoulder) where the own vehicle can stop safely, in a range inwhich recognition and movement of the recognition device 6 having nofailure are enabled. The own vehicle stops safely at the exterior of arunning traffic lane. Then, the safe state is continuously maintainedand control can be delivered to the user according to a situation.

In this case, when the dynamic object exists in a peripheral portion ofthe own vehicle such as the failure occurrence range and a trajectory toan area where the own vehicle can stand still safely and an actionprediction trajectory of the dynamic object cross, a state is not safeand the above action is not performed.

Because the recognition function is lost in the failure occurrencerange, it is difficult to decide which obstacle exists. For this reason,it is necessary to cause the own vehicle not to enter the failureoccurrence range to maintain the safe state.

As another example of the safe state, an operation is delivered to theuser in a state in which a current running state is maintained. As inthe examples of FIGS. 12(a) and 12(b), when the front side isrecognizable and a straight running state can be continuouslymaintained, notification to the user to be described below is performedwhile straight running is continuously performed and control isdelivered to the user. This case is also the safe state when the failureoccurs.

In this case, the own vehicle is drawn to a traffic lane opposite to adirection in which the failure occurs in the recognition device 6 in atraffic lane to keep away from the dynamic object existing in thefailure occurrence range and safety can be increased.

Likewise, deceleration is performed gently while a recognizable range isdetermined during delivering the operation to the user, so that a speedwhen collision occurs can be reduced, and safety can be improved.

When the front side can be recognized and a straight path is continued,it is not essential to deliver the operation to the user and the actionplanning system may execute the control continuously. The operation isdelivered to the user only when it is difficult to secure the safety ina state in which the failure occurs in the recognition device, such asentry of the dynamic object in the failure occurrence range, or it willbe difficult to secure the safety. In this way, the action planningsystem can execute long-term control and the load of the user can bealleviated.

For the priority when it is difficult to enter any safe state, thehighest priority is given to avoidance of collision in a recognizablerange and the second priority is given to collision with a predictionobstacle in the failure occurrence range. That is, when it is determinedthat it is difficult to enter any safe state, an action to avoid thecollision in the recognizable range is taken and an action plan forallowing the collision with the prediction obstacle is made. As aresult, an action to avoid collision with a recognizable vehicle can betaken.

<Output Vehicle State to User and Notify Exterior of Vehicle of VehicleState>

The vehicle control system 3 outputs a current vehicle state to the uservia the output device 7 or outputs the current vehicle state to theexterior of the vehicle via the notification device 9 or thecommunication device 3. For example, when a failure occurs in anyportion of the vehicle system 1, lighting such as a warning or a warningusing a sound is given to the user via the output device 7. In addition,an output of a warning state using a lamp, an output of a warning soundusing a speaker, or an output of information regarding the failure isgiven to the exterior of the vehicle via the notification device 9 orthe communication device 3.

When the failure occurs in the recognition device 6, occurrence of thefailure is notified to the user by a warning or a sound and a failurerange is displayed by a display or a warning light of the output device7. An output example of the output device 7 is illustrated in FIG. 13.For an output of the display, like an example illustrated in a situationdisplay 1301 of FIG. 13, a failure range is notified and a generatedtrajectory is displayed, so that a future operation of the vehicle canbe notified. As a result, the future action is easily predicted like thecase where the control is delivered to the user when the failure occursand processing can be delivered safely. As a different example, anexample of displaying only a latest direction of a trajectory simplysuch that the user easily view a future trajectory like a course display1302 or a display example of enabling a failure range to be easilyviewed like a failure place display 1303 are enabled. In this way, theuser easily assumes the behavior of the vehicle at the time ofdelivering. The display and the warning sound are output at the sametime or flickering display of video is performed, so that the user caneasily recognize display of the display device 1300.

For the notification to the exterior of the vehicle, occurrence of thefailure in the own vehicle, a failure occurrence range, and a futuretrajectory direction of the own vehicle are notified via thenotification device 9 or the communication device 3 in the same way. Inthis way, a following vehicle can predict the action of the vehiclesystem 1 in which the failure has occurred and secondary damage such ascollision with the own vehicle can be avoided.

Second Embodiment

An action planning system according to a second embodiment of thepresent invention will be described. The second embodiment is differentfrom the first embodiment in that highly safe prediction is performedfor action prediction of a dynamic object of a failure area performed byan integration recognition unit 602.

An example of the case where a failure occurs in a recognition device 6is illustrated in FIGS. 14 (a) and 14(b) Here, an example of the casewhere a failure occurs in a recognition device monitoring a right sideof a vehicle is illustrated (FIG. 14(a)). Here, the integrationrecognition unit 602 assumes that a dynamic object in a failureoccurrence range takes a high-risk action for an own vehicle. Inexamples of FIGS. 14(a) and 14(b), an action to approach an own vehiclelike the case where a vehicle of a rear right side changes a trafficlane to a left side is predicted.

By the action prediction, the own vehicle determines that there is thepossibility of collision at the time of going straight and takes anaction to move in a leftward direction. As such, prediction of thedynamic object of the failure occurrence range is determined as highrisk, so that safety of the own vehicle can be secured in many cases.

Here, for a high-risk action, an action for approaching the own vehicleis main. However, for an approaching method, a physical limit value(speed/acceleration) is assumed and trajectory generation according toaction prediction thereof is performed, so that it is possible to takean action not to collide with the own vehicle, as long as a physicalvalue is not more than the physical limit value.

As a different action prediction method, for approaching the ownvehicle, an action in a range in which the Road Traffic Law is obeyed ispredicted. For example, a speed upper limit of the dynamic object is aregulation speed or an excess of a constant ratio of the regulationspeed by a safety avoidance action, required time of a change of atraffic lane of the dynamic object is equal to or more than severalseconds including time of a signal, the dynamic object does not run onthe exterior of the traffic lane, and the dynamic object does notperform unreasonable passing. As such, action prediction is performedincluding a situation where the dynamic object is placed, so that anaction in which a vehicle obeying the Road Traffic Law does not collidewith the own vehicle can be taken, even though the dynamic object existsin the failure range.

As a different action prediction method, a learning result of the pastaction of the vehicle is used. An action of the dynamic object in thefailure range is predicted using behavior data of the vehicle observedby the own vehicle or other vehicle in the past. In this case, an actionis planned to avoid an action having the highest risk (approaching theown vehicle in short time) in the vehicle behavior data observed in thepast. As such, the action prediction is performed, so that an action inwhich the vehicle does not collide with the own vehicle can be taken, aslong as there is no vehicle taking an action having the higher risk thana vehicle pattern learned in the past, even though the dynamic objectexists in the failure range. In addition to using the worst value forthe learning result, a mode or an operation pattern of 90% in pastlearning patterns is determined as safe and the risk is high as comparedwith the prediction of the worst value, but action prediction that candeal with only a more frequent risk can be performed.

Third Embodiment

An action planning system according to a third embodiment of the presentinvention will be described. The third embodiment is different from thefirst embodiment in that a failure occurrence range is recognized and atrajectory generation/determination unit 603 executes trajectorygeneration processing for avoiding a dynamic object from entering therange by the generation of the trajectory to cope with the failure(S105).

FIGS. 15(a) and 15(b) illustrate operation examples when the failureoccurs. FIG. 15(a) illustrates an example of the case where arecognition device of a right side of a vehicle fails and the vehicleruns on a rear side of a failure occurrence range of the right side. Inthis case, according to the relative speed of an own vehicle and thevehicle of the rear right side, the vehicle may enter a failureoccurrence area. To avoid this, control is executed to increase thespeed of the own vehicle. When the safe state when the failure occurscan be maintained, the own vehicle moves in an avoidance direction. Inthis way, the vehicle can be avoided from moving in a dead angledirection and safety can be improved.

Particularly, when prediction of the safety of the operation object isperformed safely as described in the second embodiment, the possibilitythat an action range is narrowed becomes high due to existence of thedynamic object in the failure occurrence range. For this reason, by acombination with this embodiment, an action enabled range of a vehiclesystem can be widened in a state in which safety is secured.

Fourth Embodiment

An action planning system according to a fourth embodiment of thepresent invention will be described. The fourth embodiment is differentfrom the first embodiment in that a failure occurrence range isrecognized in action prediction performed by an integration recognitionunit 602 and an outside-world recognition map is updated on theassumption that a dynamic object enters the range.

FIGS. 16(a) to 16(d) illustrate operation examples when a failureoccurs. FIG. 16(a) illustrates an example of the case where arecognition device of a right side of a vehicle fails. When the dynamicobject does not exist in a failure occurrence area until just before thefailure occurs, as illustrated in FIG. 16(b), it is determined that thedynamic object does not exist in the failure occurrence area in anoutside-world recognition map, immediately after the failure occurs.After constant time passes, it is assumed that the object enters from arange in which the dynamic object can enter from an entry allowed range(a range in which it is assumed that there is no stationary object ofwhich an entry is not allowed), in an area where a failure occurs andrecognition is disabled, and prediction is performed (FIG. 16(c)). Then,similar to the embodiments, it is assumed that the dynamic object entersfrom a direction where recognition is disabled and an entry is allowedand a trajectory is generated.

In this way, when there is a dead angle direction due to the occurrenceof the failure, the action prediction is performed on the dynamic objectentering from the dead angle direction and collision can be avoided.

Fifth Embodiment

An action planning system according to a fifth embodiment of the presentinvention will be described. The fifth embodiment is different from thefirst embodiment in processing in the case where there is a recognitiondevice having a recognition range overlapping a recognition range of arecognition device where a failure has occurred.

An example of the case where one of recognition devices of a right sideof a vehicle fails is illustrated. A recognition result of a recognitiondevice in which a failure does not occur in that case is illustrated inFIG. 17(a). Here, in the recognition result of FIG. 17(a), there is theundecided area (area where recognition by the recognition device isdifficult) described above and there is an area where a dynamic objectmay exist (“undecided detection area” in the drawing). When a failuredoes not occur, the existence probability of the undecided area isimproved by superimposition of the probabilities output from a pluralityof recognition devices, as described above.

An example of the case where a failure occurs in a recognition deviceobserving the same area as the recognition area in FIG. 17(a) isillustrated in FIG. 17 (b). In this case, an outside-world recognitionmethod when the failure occurs is the same as the method described inthe first embodiment. Here, as an example, the existence probabilitythat a dynamic object exists in an area where the dynamic object doesnot exist until the failure occurs, in a failure occurrence range, isset as 0.3 uniformly and an area where the dynamic object exists untilthe failure occurs and an action prediction range of the dynamic objectthereof are set as the existence probability of 0.8.

A superimposition result of these recognition results is described inFIG. 17(c). In an area of A in the drawing, because it is recognizedthat an object does not exist (the existence probability is 0) in arecognition device (hereinafter, referred to as a normal recognitiondevice) in which a failure does not occur, the existence probability ofthe dynamic object to be a multiplication result thereof becomes 0.

In an area of B in the drawing, because an output of the normalrecognition device is an undecided area, the existence probabilitycalculated by the normal recognition device is 0.3 and if the existenceprobability is multiplied with the existence probability (0.3) of thefailure occurrence area, the existence probability of a result becomes0.09. In normal superimposition, if an output of a recognition device inwhich a failure occurs at the present time is normal and the existenceprobability is clear (1 or 0), the existence probability after thesuperimposition becomes clear more (0.3 or 0). However, when an obstacledoes not exist in the past even in the case where the failure occurs,the existence probability can be decreased.

In an area of (C) in the drawing, recognition can be surely performedfrom a normal recognition device (existence probability of 1.0). If theexistence probability is superimposed on the existence probability (0.8)of the recognition device in which the failure occurs, the existenceprobability becomes 0.8. Similar to the above, if the recognition devicein which the failure occurs is normal, the existence probability becomesclear more. However, the existence probability can be increased for aplace where the existence probability is high, on the basis ofinformation in which the dynamic object exists in the past.

As such, for a range in which recognition is performed bysuperimposition of a plurality of recognition devices, prediction isperformed on a failure range when the failure occurs and the existenceprobabilities are estimated and superimposed. As a result, an action inwhich safety according to a failure prediction result is secured can betaken as compared with the case where recognition is performed by onlythe normal recognition device when the failure occurs.

For the action prediction of the failure occurrence range at that time,the prediction is performed safely as described in the secondembodiment. As a result, the existence probability of an undecideddetection area can be predicted safely as compared with the case wherean outside-world recognition map is generated by only the recognitiondevice in which the failure does not occur.

Sixth Embodiment

An action planning system according to a sixth embodiment of the presentinvention will be described. Here, an operation example of the casewhere a non-assumed situation occurs during a traffic lane changeoperation will be described. As a specific example of the non-assumedsituation, FIG. 18(a) illustrates an example of the case where a failureoccurs in a recognition device monitoring a rear side. An example of anoutside-world recognition map in that case and a generation example of atrajectory are illustrated in FIG. 18 (b). Similar to the firstembodiment, in this embodiment, control is executed while a failurerange is predicted. However, in this case, a safe state when a failureoccurs is to stop a traffic lane change operation (that is, an operationof movement to left and right traffic lanes). As a result, the risk isavoided from occurring due to continuously executing the traffic lanechange operation, with respect to vehicles approaching from rear sidesof both the left and right traffic lanes.

At this time, a vehicle system provides the vehicle state to a user. Assuch, occurrence of a failure is provided to the user, so that the usercontinuously executes control thereafter and can maintain a safe state.In addition, the vehicle system provides the occurrence of the failureto the exterior of a vehicle. As such, the occurrence of the failure isprovided to the exterior of the vehicle, so that other vehicle canconfirm that a failure occurs in the vehicle executing the control andcan take an avoidance action. As an example of a method of providing theoccurrence of the failure to the exterior of the vehicle, a hazard lampis turned on.

As controls when the traffic lane change operation is stopped, variouscontrols are considered. First, one of the controls when the trafficlane change operation is stopped is to continuously execute an operationalong the traffic lane, in a state in which a traffic lane widthdirection position where the traffic lane change operation has stoppedis maintained. In this case, a trajectory generation/determination unit603 newly generates a trajectory along the traffic lane from theposition where the traffic lane change operation has stopped and amotion control unit 604 controls the vehicle along the generatedtrajectory. During this period, the control is delivered to the user tobe a driver. For example, in the case where the non-assumed situationoccurs when the vehicle is at the traffic lane width direction positionover a plurality of traffic lanes, the vehicle runs along the trafficlanes in a state in which the vehicle is over the plurality of trafficlanes. In the case where the non-assumed situation is failures of somerecognition devices 6, when an integration recognition unit 602 candetect a course direction (that is, a direction of the traffic lane or adirection of a dividing line such as a white line) of a road byrecognition devices 6 in which a failure does not occur, the vehicle iscontrolled on the basis of the detected course direction.

Another example of the controls when the traffic lane change operationis stopped is as follows. First, the trajectory generation/determinationunit 603 divides the traffic lane change operation into a plurality ofsteps and executes processing according to each step. It is determinedwhich step the vehicle belongs to, on the basis of a trajectorygenerated when the traffic lane change operation is executed or arelative position of the vehicle to the traffic lane. Because thetraffic lane change operation is an operation associated with anoriginal traffic lane and a different traffic lane such as a neighboringtraffic lane, operations considered as safe operations are differentaccording to the individual steps. Therefore, if different control canbe executed according to each step of the traffic lane change operationwhen the non-assumed situation occurs, the possibility of securingsafety can be increased for other vehicle as well as an own vehicle.

When the step which the vehicle belongs to is determined by the relativeposition of the vehicle to the trajectory, a progress degree or aprogress rate of the own vehicle to an entire route from a startingpoint to an ending point of the trajectory is used. In addition, whenthe step which the vehicle belongs to is determined by the relativeposition of the vehicle to the traffic lane, the step is determined onthe basis of the relative position of the vehicle to a directioncoupling a traffic lane of a change origin and a traffic lane of achange destination (that is, a direction crossing a course direction ofa road or a traffic lane width direction). In addition, it is simplestand easiest to set a position of the vehicle to a center position of alongitudinal direction and a width direction of the vehicle. However,any position such as a position of a camera, a position of a driverseat, a corner of the vehicle, and a front portion can be used.

Specifically, the traffic lane change operation is divided into threesteps of an initial step, a middle step, and a late step. The individualsteps may be obtained by dividing the traffic lane change operation intothe three steps simply. However, the initial step may be set as the casewhere the vehicle is included in only a traffic lane of a movementorigin, the middle step may be set as the case where the vehicle is overboth traffic lanes, and the late step may be set as the case where thevehicle is included in only a traffic lane of a movement destination.

Next, control in each step when the traffic lane change operation isdivided into the three steps will be described. First, a state(hereinafter, referred to the middle step) other than the initial step(for example, ⅔ of the vehicle width exists on a traffic lane before themovement) and the late step (for example, ⅔ of the vehicle width existson a traffic lane after the movement) will be described. In the initialstep, the vehicle moves to return to the traffic lane of the movementorigin and in the late step, the vehicle moves to the traffic lane ofthe movement destination. In this way, because a movement amount afterthe traffic lane change operation is stopped can be reduced, safety canbe increased. More specifically, in the initial step, the vehiclereturns to an original traffic lane width direction position (a startingpoint of the trajectory of the original traffic lane change operation)on the traffic lane of the movement origin. In addition, in the latestep, the vehicle moves to an ending point of the trajectory of theoriginal traffic lane change operation. Control of the late stepcorresponds to that the traffic lane change operation stopped once isresumed and completed as a result.

In the initial step or the late step, various other controls areconsidered. For example, a method of moving the own vehicle to aposition where a passage width of other vehicle is secured in a lane inwhich an inclusion ratio of the own vehicle is small in the two trafficlanes which the own vehicle is over, is also considered. That is, in thecase of the initial step, the own vehicle moves to a position where apassage width of other vehicle is secured in the traffic lane of themovement origin and in the case of the late step, the own vehicle movesto a position where a passage width of other vehicle is secured in thetraffic lane of the movement destination. According to this control,because it is possible to secure a space where other vehicle passesthrough the side of the own vehicle, other vehicle can avoid the riskeasily.

In the middle step, various controls are considered. Similar to one ofthe controls when the traffic lane change operation is stopped, which isdescribed above, one of the controls in the middle step is to execute anoperation along the traffic lane, in a state in which a traffic lanewidth direction position where the traffic lane change operation hasstopped is maintained.

As another example of the control in the middle step, it is consideredthat the own vehicle is moved in a lane in which an inclusion ratio ofthe own vehicle is large in the two traffic lanes which the own vehicleis over. In this way, because a movement amount after the traffic lanechange operation is stopped can be reduced, safety can be increased.More preferably, the own vehicle is moved such that a side portion ofthe own vehicle comes to a position corresponding to a boundary line(generally, a dividing line) of the two traffic lanes. In this way, theown vehicle evacuates from the other traffic lane different from thetraffic lane of the own vehicle. Meanwhile, because the own vehicle isat a position close to an edge in the traffic lane of the own vehicle,the risk avoidance can be easily performed for other vehicle running onthe same traffic lane as the own vehicle.

As another control example of the middle step, a method of measuring adistance of each following vehicle and the own vehicle when there arefollowing vehicles in both left and right neighboring traffic lanes andmoving the own vehicle to a position close to the traffic lane at alarge distance is also considered. As a different control example, amethod of moving the own vehicle to the preceding traffic lane or thetraffic lane having a running space such as a road shoulder in both thetraffic lanes is also considered. In this way, even when the followingvehicle approaches rapidly, an avoidance route of the following vehiclecan be secured.

In addition, an example of the case where a failure occurs in functionsof a part (for example, right-half recognition devices of a rear side)of recognition devices of the rear side of the vehicle is illustrated inFIG. 18(c). As such, when the safety of any traffic lane is secured, theown vehicle moves to the side of the traffic lane (in this example, theleft side) in which the safety is confirmed (FIG. 18(d)), so that a safeaction can be taken.

For the action of the middle step, the priority is changed by the courseof the vehicle and the action thereafter and the action can be taken.For example, when a fork/route change is performed by the traffic lanechange, it is considered that a threshold to continue a movement isincreased. For example, in the above, as a threshold for being dividedinto the late step, the ratio where the own vehicle exists on thetraffic lane after the movement is set as “⅔ of the vehicle width”.However, the threshold may be set as a value smaller than ⅔. Inaddition, in the middle step, it is considered that, even when thetraffic lane width direction position is maintained and the own vehicleruns, the own vehicle is maintained to be close to the movementdestination traffic lane and the traffic lane change is easily performedthereafter. In the case of a front vehicle passing operation, it isconsidered that a threshold of the original traffic lane return isincreased. For example, in the above, as a threshold for being dividedinto the initial step, the ratio where the own vehicle exists on thetraffic lane after the movement is set as “⅔ of the vehicle width”.However, the threshold may be set as a value smaller than ⅔.

The sixth embodiment is an embodiment of the case where the recognitiondevice 6 fails as the non-assumed situation. However, the control in thecase where the traffic lane change operation is stopped in the sixthembodiment is effective to not only the case where the failure occurs inthe recognition device 6 but also other case. For example, other case isthe case where other vehicle (particularly, a following vehicle) notassumed when a trajectory is generated during the traffic lane changeoperation is recognized. Even in other case, the traffic lane changeoperation is stopped, so that occurrence of the risk can be avoided.Among the configurations and the effects described above, theconfiguration and the effect not limited to the case of the occurrenceof the failure are commonly applicable to other case.

According to the embodiments described above, when a failure occurs in arecognition device of a vehicle system, a failure occurrence range isrecognized and an action according to the failure occurrence range istaken, so that safety of the vehicle system can be secured.Particularly, a highly safe trajectory in which a safe state ismaintained can be generated using past outside-world recognitioninformation of the failure occurrence range.

In addition, according to a different embodiment, trajectory generationis performed after safely performing prediction of a dynamic objectexisting before occurrence of the failure in the failure occurrencerange. As a result, safe action planning can be made and a safe statecan be maintained.

In addition, according to a different embodiment, a trajectory toprevent the dynamic object from entering the failure occurrence range isgenerated and an action is taken, so that the risk of the dynamic objectentering the failure occurrence range can be avoided.

In addition, according to a different embodiment, action prediction isperformed on the assumption that the dynamic object enters from a deadangle direction of the failure occurrence range, so that a safetrajectory can be generated with respect to the dynamic object enteringfrom the dead angle direction of the failure occurrence range.

In addition, according to a different embodiment, when a failure occursin any recognition device in an area where a plurality of recognitiondevices perform recognition by a superimposition method, the dynamicobject is recognized with high precision in accordance with an actionprediction result and action planning when the failure occurs can bemade.

In addition, according to a different embodiment, even in the case wherethe failure occurs in a recognition device when a traffic lane changes,maintenance of a safe state according to a current control state andnotification to a user and the exterior of a vehicle are performed andthe safe state can be maintained.

REFERENCE SIGNS LIST

-   1 vehicle system-   2 vehicle control system-   3 communication device-   4 vehicle control system-   5 drive device-   6 recognition device-   7 output device-   8 input device-   9 notification device-   301 network link-   302 ECU-   303 GW-   401 processor-   402 I/O-   403 timer-   404 ROM-   405 RAM-   406 internal bus-   501 control unit-   502 communication management unit-   503 time management unit-   504 data table-   505 buffer-   601 action planning system-   602 integration recognition unit-   603 trajectory generation/determination unit-   604 motion control unit-   605 user input unit-   606 output management unit-   607 notification management unit-   608 failure detection unit-   1001 outside-world recognition map-   1300 display device-   1301 situation display-   1302 course display-   1303 failure place display

1. An action planning device, comprising: a failure detection unit whichdetects occurrence of a failure in outside-world recognition units andperforms notification; an integration recognition unit which integratesoutside-world recognition information output from the outside-worldrecognition units and outputs integration recognition information; and atrajectory generation/determination unit which sends, to a drive unit, acontrol instruction to execute an action on the occurrence of thefailure, on the basis of the integration recognition information outputby the integration recognition unit, wherein the trajectorygeneration/determination unit sends an instruction to take an action tocope with the failure, on the basis of the notification of the failuredetection unit.
 2. The action planning device according to claim 1,wherein the trajectory generation/determination unit generates atrajectory to cope with the failure to enter a safe state, on the basisof past outside-world recognition information in a failure occurrencerange notified by the failure detection unit.
 3. The action planningdevice according to claim 2, wherein the trajectorygeneration/determination unit determines failure occurrence time anddiscards outside-world recognition information after the failureoccurrence time in the failure occurrence range recognized by thefailure detection unit.
 4. The action planning device according to claim1, further comprising: a notification unit which notifies the exteriorof a vehicle of the occurrence of the failure, wherein the notificationunit notifies of failure detection information notified by the failuredetection unit, in the action to cope with the failure.
 5. The actionplanning device according to claim 2, wherein the integrationrecognition unit predicts an action of a dynamic object in the failureoccurrence range safely.
 6. The action planning device according toclaim 2, wherein the trajectory generation/determination unit generatesa trajectory to avoid a dynamic object from entering the failureoccurrence range.
 7. The action planning device according to claim 2,wherein the integration recognition unit superimposes an actionprediction result of the failure occurrence range and the existenceprobability of outside-world recognition information of the same areaoutput by a recognition device in which a failure does not occur andexecutes an operation.
 8. The action planning device according to claim2, wherein the trajectory generation/operation unit maintains a middlestate of a traffic lane as a safe state in a middle state of a change ofthe traffic lane.
 9. The action planning device according to claim 1,further comprising: an output unit which notifies a user of a failureoccurrence state, wherein the output unit notifies the user of theoccurrence of the failure, when the failure occurs.